Papers with linear programming solver
QualEval: Qualitative Evaluation for Model Improvement (2024.naacl-long)
Copied to clipboard
Vishvak Murahari, Ameet Deshpande, Peter Clark, Tanmay Rajpurohit, Ashish Sabharwal, Karthik Narasimhan, Ashwin Kalyan
| Challenge: | Quantitative evaluation metrics are inadequate for large language models due to complexity of tasks and cannot provide actionable diagnostics. |
| Approach: | They propose a quantitative evaluation tool called QualEval that uses automated qualitative evaluation as a vehicle for model improvement. |
| Outcome: | The proposed method improves the performance of the Llama 2 model by 15% compared to baselines. |